Mind the Gap: Machine Translation by Minimizing the Semantic Gap in Embedding Space

The conventional statistical machine translation (SMT) methods perform the decoding process by compositing a set of the translation rules which are associated with high probabilities. However, the probabilities of the translation rules are calculated only according to the cooccurrence statistics in the bilingual corpus rather than the semantic meaning similarity. In this paper, we propose a Recursive Neural Network (RNN) based model that converts each translation rule into a compact real-valued vector in the semantic embedding space and performs the decoding process by minimizing the semantic gap between the source language string and its translation candidates at each state in a bottom-up structure. The RNN-based translation model is trained using a max-margin objective function. Extensive experiments on Chinese-to-English translation show that our RNN-based model can significantly improve the translation quality by up to 1.68 BLEU score. Introduction The conventional statistical machine translation (SMT) models, such as phrase-based models (Koehn et al. 2007), formal syntax-based models (Chiang 2007; Xiong, Liu, and Lin 2006) and linguistically syntax-based models (Liu, Liu, and Lin 2006; Huang, Knight, and Joshi 2006; Galley et al. 2006; Zhang et al. 2008), perform the decoding process and generate the translation result by compositing a set of translation rules which are associated with high probabilities. The probabilities of the translation rules (e.g. the phrasal translation probabilities and the lexical weights in phrase-based and formal syntax-based models) are all computed based on the cooccurrence statistics of the rule’s sourceand targetsides in the bilingual corpus. However, the cooccurrence statistics is much biased to the bilingual corpus and is not sufficient to show whether the sourceand target-sides in a translation rule are in the same meaning, especially for the low frequent but correct translation rules. Accordingly, the conventional SMT models cannot guarantee that the generated translations are in the most similar semantic meanings with the source-side inputs. Copyright c 2014, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. developing 发展 the relations between 之间 的 关系 the two countries 两 国 the relations between the two countries 两 国 之间 的 关系 developing the relations between the two countries 发展 两 国 之间 的 关系